Papers with resource efficiency
Learning to Verify Summary Facts with Fine-Grained LLM Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly enhanced the text summarization performance, but hallucination issues still occur in summaries. |
| Approach: | They propose a large-scale dataset containing fine-grained factual feedback on summaries that can be fine tuned by using Large Language Models (LLMs) they employ 10 distinct LLMs for diverse summary generation and Llama-3-70B-Instruct for feedback. |
| Outcome: | The proposed model outperforms models trained on smaller human-annotated datasets while maintaining high performance. |
Beyond Grid Search: Leveraging Bayesian Optimization for Accelerating RAG Pipeline Optimization (2026.eacl-industry)
Copied to clipboard
| Challenge: | Finding optimal configurations via grid search is computationally prohibitive, limiting real-world scalability. |
| Approach: | They compare BO with grid search to find optimal configurations for RAG pipelines . their framework explores global pipeline and local component-wise optimization . |
| Outcome: | The proposed approach reduces optimization time by up to 84% while maintaining comparable accuracy. |
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent evidence shows that large-size pre-trained language models do not satisfy the logical negation property (LNP) However, their reliability is being challenged due to faulty behaviours and incomprehension on number-related representations. |
| Approach: | They propose a new intermediate training task to directly learn meaning text correspondence instead of relying on the distributional hypothesis. |
| Outcome: | The proposed approach outperforms previous models on 7 GLUE tasks and outperformed previous models. |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |
CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models (LLMs) focus on task completion, but neglect a crucial capability: the ability to devise and adjust cost-optimal plans in response to changing environments. |
| Approach: | They propose a scalable, cost-centric benchmark to evaluate agents’ economic reasoning and replanning abilities. |
| Outcome: | Evaluating leading open-sourced and proprietary models on CostBench reveals a substantial gap in cost-aware planning . |
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt optimization methods rely on extensive manual effort or meta-cognitive abilities, making them less effective for LwLLMs. |
| Approach: | They propose a direct behavior optimization parameter that transforms the optimization of complex prompts into discrete, quantifiable execution sequences using a gradient-free Monte Carlo Tree Search. |
| Outcome: | The proposed method outperforms current prompt optimization methods on seven challenging tasks where state-of-the-art LLMs excel but LwLLMs generally underperform. |
Evaluating Parameter-Efficient Finetuning Approaches for Pre-trained Models on the Financial Domain (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale language models with millions, billions, or trillions of trainable parameters are becoming increasingly popular. |
| Approach: | They compare performance of financial BERT-like models to their fully fine-tuned counterparts by using parameter-efficient tuning methods. |
| Outcome: | The proposed approaches match full fine-tuning performance on common NLP tasks, but are less studied in finance. |
SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | SLMs offer computational efficiency and accessibility, yet a systematic evaluation of their performance and environmental impact remains lacking. |
| Approach: | SLM-Bench evaluates 15 SLMs on 9 NLP tasks using 23 datasets . compared accuracy, computational efficiency, and sustainability metrics . |
| Outcome: | SLM-Bench evaluates 15 SLMs on 9 NLP tasks using 23 datasets spanning 14 domains. |
GottBERT: a pure German Language Model (2024.emnlp-main)
Copied to clipboard
Raphael Scheible, Johann Frei, Fabian Thomczyk, Henry He, Patric Tippmann, Jochen Knaus, Victor Jaravine, Frank Kramer, Martin Boeker
| Challenge: | Pre-trained language models have advanced natural language processing (NLP) despite the introduction of BERT, single-language models are still relevant. |
| Approach: | They present a German singlelanguage RoBERT model pre-trained exclusively on the German portion of the OSCAR dataset. |
| Outcome: | The GottBERT model outperforms the existing models on Named Entity Recognition and text classification tasks. |
ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to chain-of-thought reasoning incur high inference latency due to long generation traces. |
| Approach: | They propose a confidence-gated cascaded verification framework that reduces the trade-off between generation and verification. |
| Outcome: | The proposed framework achieves 2.24 speedups while matching target-model accuracy. |